{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Pandas 相关性分析 - corr() 和 cov()\n",
        "\n",
        "本教程详细介绍如何使用 `corr()` 和 `cov()` 方法进行相关性分析。\n",
        "\n",
        "## 目录\n",
        "1. corr() - 相关系数\n",
        "2. cov() - 协方差\n",
        "3. 相关系数的计算方法\n",
        "4. 相关性分析的应用\n",
        "5. 相关性矩阵可视化\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 导入库\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import warnings\n",
        "warnings.filterwarnings('ignore')\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 1. corr() - 相关系数\n",
        "\n",
        "### 方法说明\n",
        "\n",
        "`corr()` 计算列之间的相关系数（默认使用 Pearson 相关系数）。\n",
        "\n",
        "**语法:**\n",
        "```python\n",
        "df.corr(method='pearson', min_periods=1, numeric_only=None)\n",
        "```\n",
        "\n",
        "**主要参数:**\n",
        "- `method`: 计算方法，'pearson', 'kendall', 'spearman'，默认 'pearson'\n",
        "- `min_periods`: 计算所需的最小样本数\n",
        "- `numeric_only`: 是否只计算数值型列，默认 None\n",
        "\n",
        "**相关系数范围:** -1 到 1\n",
        "- **1**: 完全正相关\n",
        "- **0**: 无相关性\n",
        "- **-1**: 完全负相关\n",
        "\n",
        "**适用场景:** 特征选择、变量关系分析、数据探索\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例1: 创建示例数据\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "np.random.seed(42)\n",
        "df = pd.DataFrame({\n",
        "    '年龄': np.random.randint(20, 60, 100),\n",
        "    '身高': np.random.normal(170, 10, 100),\n",
        "    '体重': np.random.normal(70, 10, 100),\n",
        "    '工资': np.random.normal(8000, 2000, 100),\n",
        "    '工作经验': np.random.randint(0, 20, 100)\n",
        "})\n",
        "\n",
        "# 创建一些相关性\n",
        "df['工资'] = df['工作经验'] * 500 + df['工资'] * 0.5  # 工资与工作经验相关\n",
        "df['体重'] = (df['身高'] - 150) * 0.8 + np.random.normal(0, 5, 100)  # 体重与身高相关\n",
        "\n",
        "print(\"原始数据 (前10行):\")\n",
        "print(df.head(10))\n",
        "print(\"\\n数据形状:\", df.shape)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例2: 计算相关系数矩阵\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 计算相关系数矩阵（默认Pearson方法）\n",
        "correlation_matrix = df.corr()\n",
        "print(\"相关系数矩阵 (Pearson):\")\n",
        "print(correlation_matrix)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例3: 计算单个列的相关系数\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 计算工资与其他变量的相关系数\n",
        "print(\"工资与其他变量的相关系数:\")\n",
        "print(df.corr()['工资'].sort_values(ascending=False))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 2. cov() - 协方差\n",
        "\n",
        "### 方法说明\n",
        "\n",
        "`cov()` 计算列之间的协方差。\n",
        "\n",
        "**语法:**\n",
        "```python\n",
        "df.cov(min_periods=None, numeric_only=None)\n",
        "```\n",
        "\n",
        "**主要参数:**\n",
        "- `min_periods`: 计算所需的最小样本数\n",
        "- `numeric_only`: 是否只计算数值型列，默认 None\n",
        "\n",
        "**协方差特点:**\n",
        "- 协方差不受标准化影响，其值范围不受限制\n",
        "- 相关系数 = 协方差 / (标准差1 × 标准差2)\n",
        "- 协方差衡量变量间的关系强度和方向\n",
        "\n",
        "**适用场景:** 风险评估、投资组合分析\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例4: 计算协方差矩阵\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 计算协方差矩阵\n",
        "covariance_matrix = df.cov()\n",
        "print(\"协方差矩阵:\")\n",
        "print(covariance_matrix)\n",
        "print(\"\\n说明: 协方差不受标准化影响，值范围不受限制\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 3. 相关系数的计算方法\n",
        "\n",
        "### 示例5: 不同的相关系数计算方法\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# Pearson 相关系数（线性相关性，默认）\n",
        "pearson_corr = df.corr(method='pearson')\n",
        "print(\"Pearson 相关系数 (线性相关性):\")\n",
        "print(pearson_corr['工资']['工作经验'])\n",
        "\n",
        "# Spearman 相关系数（秩相关性，适用于非线性关系）\n",
        "spearman_corr = df.corr(method='spearman')\n",
        "print(\"\\nSpearman 相关系数 (秩相关性):\")\n",
        "print(spearman_corr['工资']['工作经验'])\n",
        "\n",
        "# Kendall 相关系数（秩相关性，更稳健）\n",
        "kendall_corr = df.corr(method='kendall')\n",
        "print(\"\\nKendall 相关系数 (秩相关性，更稳健):\")\n",
        "print(kendall_corr['工资']['工作经验'])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 4. 相关性分析的应用\n",
        "\n",
        "### 示例6: 查找高相关性的特征\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 查找高度相关的特征对（相关性 > 0.5）\n",
        "corr_matrix = df.corr()\n",
        "high_corr_pairs = []\n",
        "\n",
        "for i in range(len(corr_matrix.columns)):\n",
        "    for j in range(i+1, len(corr_matrix.columns)):\n",
        "        col1 = corr_matrix.columns[i]\n",
        "        col2 = corr_matrix.columns[j]\n",
        "        corr_value = corr_matrix.iloc[i, j]\n",
        "        if abs(corr_value) > 0.5:\n",
        "            high_corr_pairs.append((col1, col2, corr_value))\n",
        "\n",
        "print(\"高相关性特征对 (|相关系数| > 0.5):\")\n",
        "for col1, col2, corr in sorted(high_corr_pairs, key=lambda x: abs(x[2]), reverse=True):\n",
        "    print(f\"{col1} - {col2}: {corr:.3f}\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### 示例7: 相关性矩阵可视化\n",
        "\n",
        "使用热力图可视化相关性矩阵。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "# 使用matplotlib和seaborn可视化相关性矩阵\n",
        "try:\n",
        "    import matplotlib.pyplot as plt\n",
        "    import seaborn as sns\n",
        "    \n",
        "    plt.figure(figsize=(10, 8))\n",
        "    sns.heatmap(corr_matrix, annot=True, fmt='.2f', cmap='coolwarm', \n",
        "                center=0, square=True, linewidths=1)\n",
        "    plt.title('相关性矩阵热力图', fontsize=14, fontweight='bold')\n",
        "    plt.tight_layout()\n",
        "    plt.show()\n",
        "    print(\"相关性矩阵热力图已显示\")\n",
        "except ImportError:\n",
        "    print(\"请安装matplotlib和seaborn以进行可视化\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## 总结\n",
        "\n",
        "**相关性分析方法总结:**\n",
        "1. ✅ **corr()**: 计算相关系数矩阵（默认Pearson方法）\n",
        "2. ✅ **cov()**: 计算协方差矩阵\n",
        "3. ✅ **Pearson**: 线性相关性（默认方法）\n",
        "4. ✅ **Spearman**: 秩相关性（适用于非线性关系）\n",
        "5. ✅ **Kendall**: 秩相关性（更稳健）\n",
        "\n",
        "**最佳实践:**\n",
        "- 使用 `corr()` 了解变量间的关系\n",
        "- 使用 `cov()` 了解变量间的协方差\n",
        "- Pearson适用于线性关系，Spearman/Kendall适用于非线性关系\n",
        "- 通过相关性矩阵识别冗余特征\n",
        "- 使用热力图可视化相关性矩阵\n",
        "- 注意：相关性不等于因果关系\n"
      ]
    }
  ],
  "metadata": {
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 2
}
